Usama Wazeer has written a program called the
Multiple Link Eliminator, located in
Matt's Script Archive EXTRAS!, which will take
your links.html page and take out any duplicate
links.
It is written in C and there are only two
variables you must modify: the location of the
links.html file and the file you want it to output
the modified links page to. This is a very useful
program.